The present application relates generally to an improved data processing apparatus and method and more specifically to mechanisms for high speed large scale dictionary matching.
Dictionary matching is a commonly used operation in a variety of applications. Dictionary matching is used in applications, such as text analytics, network intrusion detection systems, dictionary-based compression, virus scanners, spam filters, etc. Dictionary matching is challenging when considering large dictionary sizes and the requirement to perform wire-speed matching in real time on fast links (e.g., 10 Gbps Ethernet, 6 Gbps serial attached small computer systems interface, etc.)
Several approaches have been proposed for hardware acceleration of string matching. Prior art approaches usually employ deterministic finite automata (DFA) based algorithms. These implementations are either constrained by dictionary size or maximum achievable throughput. Some prior art approaches achieve higher bandwidth by using multiple input streams in parallel.